Unmanned Aerial Vehicles (UAV) or other manned aircraft can fly over areas of interest and make video images of those areas. Such an aerial surveillance has both military and civilian applications in the areas of reconnaissance, security, land management and natural disaster assessment to name a few. The heavily overlapped video imagery produced by the surveillance typically may be transmitted to a ground station where the images can be viewed. However, this heavily overlapped imagery shows only small pieces of a larger area of interest and in that respect is similar to the pieces of a jigsaw puzzle. Until all the pieces are put together in context, the meaning of any individual piece may be misunderstood or unclear. Therefore, a mosaic of the overlapping imagery data is needed. Prior art uses various approaches to merge such overlapping imagery data into an overall mosaic for further use and analysis.
Photogrammetrists mosaic images, but these images are typically orthorectified before the final mosaic is produced. Orthorectification, the process of converting oblique pairs of images into a single corrected top down view, requires stereo images or at least two images taken from different angles using a very high-resolution camera under strictly controlled conditions. Moreover, photogrammetry can be a labor-intensive task, requiring a human operator to place control points in the images prior to further processing. Conversely, NASA has provided image mosaics of planets, the moon and solar systems. Some newer techniques involve wavelet decomposition with equidistant measurement; whereas, older systems refer to more classical photogrammetry approaches to image mosaicking. All of the aforementioned prior art approaches are computation intensive and require extensive data collection.
Others in the field generate what are commonly known as, “Waterfall Displays” with each new image pasted at the end of a strip of the past several images. Old images roll off one end as new images are pasted on the other end. These images are not integrated in any way but are somewhat geo-referenced because one frame tends to be adjacent to the next frame in space. Nevertheless, a Waterfall Display is not a mosaic; it is just a collection of adjacent frames of video. Still others attempt to combine the images using the Optical Flow technique, which combines images based on detected image content but does not utilize geo-referencing information. Still others attempt to merge the pictures without proper integration; instead, the latest image is glued on top of whatever images came before, thereby, losing all the information from the previous images. In this case, the adjacent image edges are not blended resulting in a crude paste up appearance.
These traditional mosaicking techniques as discussed above do not provide a method for rapidly merging heavily overlapped imagery data utilizing geo-referencing information, without extensive data collection or computation.